Distributed Data Management Services for Dynamic Data Grids

Data grids are middleware systems that enable users and applications to locate, access, and place large numbers of data sets in geographically distributed storage sites. In most existing and deployed grid systems however, control of the resources is centralized and usually handled by system administrators. Such configurations hinder dynamic and scalable expansion of the Grid infrastructure and resources. We propose a new lightweight distributed, adaptive, and scalable data Grid middleware that provides transparent, fast, and reliable access to data and storage resources in data grids. At the core of our approach are dynamic data and replica location and placement techniques that adapt replica location and access to the continuously changing network connectivity and users behavior. The system is fully distributed and self configuring. In this paper we present the design of the system, the algorithms we use to implement the data management services, and demonstrate the scalability and performance of the overall system.